Randomized Heuristics for Exploiting Jacobian Scarcity

We describe a code transformation technique that, given code for a vector function F , produces code suitable for computing collections of Jacobian-vector products F (x)ẋ or Jacobiantranspose-vector products F (x) ȳ. Exploitation of scarcity a measure of the degrees of freedom in the Jacobian matrix means solving a combinatorial optimization problem that is believed to be hard. Our heuristics transform the computational graph for F , producing, in the form of a transformed graph G, a representation of the Jacobian F (x) that is both concise and suitable for evaluating large collections of Jacobian-vector products or Jacobiantranspose-vector products. Our heuristics are randomized and compare favorably in all cases with the best known heuristics.